ABSTRACT
Purpose The COVID-19 pandemic has drastically disrupted global healthcare systems. With the higher demand for healthcare and misinformation related to COVID-19, there is a need to explore alternative models to improve communication. Artificial Intelligence (AI) and Natural Language Processing (NLP) have emerged as promising solutions to improve healthcare delivery. Chatbots could fill a pivotal role in the dissemination and easy accessibility of accurate information in a pandemic. In this study, we developed a multi-lingual NLP-based AI chatbot, DR-COVID, which responds accurately to open-ended, COVID-19 related questions. This was used to facilitate pandemic education and healthcare delivery. Methods First, we developed DR-COVID with an ensemble NLP model on the Telegram platform (https://t.me/drcovid_nlp_chatbot). Second, we evaluated various performance metrics. Third, we evaluated multi-lingual text-to-text translation to Chinese, Malay, Tamil, Filipino, Thai, Japanese, French, Spanish, and Portuguese. We utilized 2,728 training questions and 821 test questions in English. Primary outcome measurements were (A) overall and top 3 accuracies;(B) Area Under the Curve (AUC), precision, recall, and F1 score. Overall accuracy referred to a correct response for the top answer, whereas top 3 accuracy referred to an appropriate response for any one answer amongst the top 3 answers. AUC and its relevant matrices were obtained from the Receiver Operation Characteristics (ROC) curve. Secondary outcomes were (A) multi-lingual accuracy;(B) comparison to enterprise-grade chatbot systems. The sharing of training and testing datasets on an open-source platform will also contribute to existing data. Results Our NLP model, utilizing the ensemble architecture, achieved overall and top 3 accuracies of 0.838 [95% confidence interval (CI): 0.826–0.851] and 0.922 [95% CI: 0.913–0.932] respectively. For overall and top 3 results, AUC scores of 0.917 [95% CI: 0.911–0.925] and 0.960 [95% CI: 0.955–0.964] were achieved respectively. We achieved multi-linguicism with nine non-English languages, with Portuguese performing the best overall at 0.900. Lastly, DR-COVID generated answers more accurately and quickly than other chatbots, within 1.12–2.15 s across three devices tested. Conclusion DR-COVID is a clinically effective NLP-based conversational AI chatbot, and a promising solution for healthcare delivery in the pandemic era.
ABSTRACT
Purpose: The COVID-19 pandemic has drastically disrupted global healthcare systems. With the higher demand for healthcare and misinformation related to COVID-19, there is a need to explore alternative models to improve communication. Artificial Intelligence (AI) and Natural Language Processing (NLP) have emerged as promising solutions to improve healthcare delivery. Chatbots could fill a pivotal role in the dissemination and easy accessibility of accurate information in a pandemic. In this study, we developed a multi-lingual NLP-based AI chatbot, DR-COVID, which responds accurately to open-ended, COVID-19 related questions. This was used to facilitate pandemic education and healthcare delivery. Methods: First, we developed DR-COVID with an ensemble NLP model on the Telegram platform (https://t.me/drcovid_nlp_chatbot). Second, we evaluated various performance metrics. Third, we evaluated multi-lingual text-to-text translation to Chinese, Malay, Tamil, Filipino, Thai, Japanese, French, Spanish, and Portuguese. We utilized 2,728 training questions and 821 test questions in English. Primary outcome measurements were (A) overall and top 3 accuracies; (B) Area Under the Curve (AUC), precision, recall, and F1 score. Overall accuracy referred to a correct response for the top answer, whereas top 3 accuracy referred to an appropriate response for any one answer amongst the top 3 answers. AUC and its relevant matrices were obtained from the Receiver Operation Characteristics (ROC) curve. Secondary outcomes were (A) multi-lingual accuracy; (B) comparison to enterprise-grade chatbot systems. The sharing of training and testing datasets on an open-source platform will also contribute to existing data. Results: Our NLP model, utilizing the ensemble architecture, achieved overall and top 3 accuracies of 0.838 [95% confidence interval (CI): 0.826-0.851] and 0.922 [95% CI: 0.913-0.932] respectively. For overall and top 3 results, AUC scores of 0.917 [95% CI: 0.911-0.925] and 0.960 [95% CI: 0.955-0.964] were achieved respectively. We achieved multi-linguicism with nine non-English languages, with Portuguese performing the best overall at 0.900. Lastly, DR-COVID generated answers more accurately and quickly than other chatbots, within 1.12-2.15 s across three devices tested. Conclusion: DR-COVID is a clinically effective NLP-based conversational AI chatbot, and a promising solution for healthcare delivery in the pandemic era.
Subject(s)
COVID-19 , Deep Learning , Humans , Natural Language Processing , Artificial Intelligence , Pandemics , IndiaABSTRACT
BACKGROUND: The 2019 novel COVID-19 has severely burdened the health care system through its rapid transmission. Mobile health (mHealth) is a viable solution to facilitate remote monitoring and continuity of care for patients with COVID-19 in a home environment. However, the conceptualization and development of mHealth apps are often time and labor-intensive and are laden with concerns relating to data security and privacy. Implementing mHealth apps is also a challenging feat as language-related barriers limit adoption, whereas its perceived lack of benefits affects sustained use. The rapid development of an mHealth app that is cost-effective, secure, and user-friendly will be a timely enabler. OBJECTIVE: This project aimed to develop an mHealth app, DrCovid+, to facilitate remote monitoring and continuity of care for patients with COVID-19 by using the rapid development approach. It also aimed to address the challenges of mHealth app adoption and sustained use. METHODS: The Rapid Application Development approach was adopted. Stakeholders including decision makers, physicians, nurses, health care administrators, and research engineers were engaged. The process began with requirements gathering to define and finalize the project scope, followed by an iterative process of developing a working prototype, conducting User Acceptance Tests, and improving the prototype before implementation. Co-designing principles were applied to ensure equal collaborative efforts and collective agreement among stakeholders. RESULTS: DrCovid+ was developed on Telegram Messenger and hosted on a cloud server. It features a secure patient enrollment and data interface, a multilingual communication channel, and both automatic and personalized push messaging. A back-end dashboard was also developed to collect patients' vital signs for remote monitoring and continuity of care. To date, 400 patients have been enrolled into the system, amounting to 2822 hospital bed-days saved. CONCLUSIONS: The rapid development and implementation of DrCovid+ allowed for timely clinical care management for patients with COVID-19. It facilitated early patient hospital discharge and continuity of care while addressing issues relating to data security and labor-, time-, and cost-effectiveness. The use case for DrCovid+ may be extended to other medical conditions to advance patient care and empowerment within the community, thereby meeting existing and rising population health challenges.
ABSTRACT
To curb the spread of the COVID-19 virus, the use of face masks such as disposable surgical masks and N95 respirators is being encouraged and even enforced in some countries. The widespread use of masks has resulted in global shortages and individuals are reusing them. This calls for proper disinfection of the masks while retaining their protective capability. In this study, the killing efficiency of ultraviolet-C (UV-C) irradiation, dry heat, and steam sterilization against bacteria (Staphylococcus aureus), fungi (Candida albicans), and nonpathogenic virus (Salmonella virus P22) is investigated. UV-C irradiation for 10 min in a commercial UV sterilizer effectively disinfects surgical masks. N95 respirators require dry heat at 100 °C for hours while steam treatment works within 5 min. To address the question on safe reuse of the disinfected masks, their bacteria filtration efficiency, particle filtration efficiency, breathability, and fluid resistance are assessed. These performance factors are unaffected after 5 cycles of steam (10 min per cycle) and 10 cycles of dry heat at 100 °C (40 min per cycle) for N95 respirators, and 10 cycles of UV-C irradiation for surgical masks (10 min per side per cycle). These findings provide insights into formulating the standard procedures for reusing masks without compromising their protective ability.